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Abstract 

Growth in satellite mobile communications leads to 
increasing requirements for high data rate transmis- 
sion that can be met by more efficient modulation 
schemes (M > 8). The 16-PSK trellis coded mod- 
ulation technique is a very promising solution. In 
this paper, a class of new 16-PSK trellis codes with 
improved error rate are designed based on the cri- 
teria on fading channels. 

1 Introduction 

The expansion of satellite communications implies 
the use of higher radio frequencies (SHF and EHF 
regions where a major disadvantage associated with 
such systems is the signal attenuation due to fading. 
Most of the TCM codes constructed for Gaussian 
channels suffer significantly degradation on fading 
channels. It has been pointed out[2][3][4] that the 
codes for fading channels should be designed by us- 
ing the criteria different from those for Gaussian 
channels. 

Some efforts have been made to constructing new 8- 
PSK TCM codes for fading channels[3]. The growth 
in satellite communications, however, leads to in- 
creasing requirements for high data rate transmis- 
sion that can be met by more efficient modulation 
schemes (M > 8). So far, the efforts on construction 
of 16-PSK TCM codes were concentrated on Gaus- 
sian channels [1][5][6][7], The problem of designing 
16-PSK TCM codes for fading channels is studied in 
this paper. A class of new 16-PSK t rellis codes with 
improved performance on fading channels are pro- 
posed based on the code design criteria for fading 
channels. The performances of the 16-PSK TCM 
systems on Rayleigh fading channels are evaluated 
both theoretically and by Monte Carlo simulations. 


2 System Description 

In the system under consideration, the digital 
source generates data digit stream at a rate of 1/Tb- 
Fro the 3-input information sequence, the convolu- 
tional encoder generate 4-output coded sequence. 
The coded sequences are then interleaved and used 
to modulate 16-PSK waveform at a rate l/3Tb sym- 
bols/s according to symbol mapping rules. The 
channel is described by a slowly varying frequency 
non-selective Rayleigh fading. The receiver consists 
of a coherent demodulator followed bv the deinter- 
leaver and a Viterbi algorithm. 

The channel is modelled by non-selective Rayleigh 
fading. We shall restrict attention to channels with 
slow amplitude variations relative to an elementary 
signaling interval and assume perfect phase tracking 
of the phase perturbation process. The interleaving 
depth is chosen to achieve ideal interleaving for a 
given fade rate in the simulations and assumed infi- 
nite in the analysis. In this model, the amplitude of 
the envelope of the received signal is multiplied by 
a random variable a, normalized so that E[a 2 ] = 1 
to ensure that the received signal energy is equal to 
the transmitted signal energy E s . The probability 
distribution function of a is 


p(a) = 2a e a 

The mean of the random variable a is 


rm 


v/? 

2 


while the variance is given by 


2 i 2 

oi - 1 - mf 
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3 Error Probability and Code 
Design Criteria 

Figure 1 shows a 16-PSK signal constellation and 
the notation where A, denotes the Euclidean dis- 
tance between channel symbols. Let C denote the 
set of all possible coded signal sequences {zn}, 
where zn is a coded symbol sequence of length N: 

ZN = (zi,Z 2 ,...,Zi,...,ZN) 

Let us suppose that v^: 

VN = (vi,V 2 j ...,Vi,...,V N ) 

an element of C, is the transmitted sequence. Then 
represents the MPSK symbol transmitted at time 
i. The corresponding received sequence at the input 
of the decoder is: 

tn = (ri , r 2 , . . . , rj, . . . , r N ) 

At time i, the received symbol r, is related to the 
transmitted symbol v t by: 

ri = aiVj + nj 

where a, is a complex random variable equal to the 
envelope of the fading channel normalized to the 
transmitted signal and n, is a sample of a zero mean 
complex additive white Gaussian noise (AWGN) 
with variance a 2 in each signal space coordinate, 
The corresponding received sequence tn can be rep- 
resented as 

r N = a N X v N + nN 
where X denotes vector multiplication. 

In the Viterbi algorithm, the path metric m(rN,ZN) 
is the sum of the branch metrics: 

N 

m(r N ,z N ) = ^m(ri,Zj) 
i=l 

The branch metric used in the algorithm is equal to 
the squared Euclidean distance: 

rnj = |rj - Zi| 2 (4) 

Definitions 

In the context of fading communications, besides 
the minimum Euclidean distance, d e , and the num- 
ber of paths at distance d € from the correct path, 


N(d e )i the following parameters play important 
roles: 

1) Effective Code Length (ECL): the minimum ef- 
fective length taken over all error paths in C where 
the Effective Length , denoted by of an error path 
is the number of erroneous symbols on the error 
path, and can be expressed as 

M/2-1 

6 = Y. w i ( 5 ) 

i=0 

where M is the number of signal points in the sig- 
nal constellation and are the weights associated 
with the corresponding nonzero symbol distances 
A t along the error path. The ECL of a TCM code 
is denoted by m this paper. 

2) Minimum Product Distance : the minimum prod- 
uct distance taken over all paths with the ECL 
where Product Distance , denoted by Ap, is the 
product of the corresponding nonzero squared sym- 
bol distances along the error path and is given by 

M/2-1 

a p = n A ? wi (6) 

i— 0 

The minimum PD is denoted by d p in this paper. 

Similar to N(d e ) y we have N(d p ) associated with 
d p , which is the number of the paths at product dis- 
tance dp from the correct path, In this paper both 
N(d e ) and N(d p ) are treated as average numbers in 
the same way a s in [7] and [10] where the average 
is taken over all paths in the trellis. 

Pairwise Error Probability 

The pairwise error probability of choosing the path 
zn instead of vn (first error event probability) is 
equal to 

P(v N — zn) = Pr{m(r N ,z N ) < m(r N ,v N )} 
or equivalently 

P(v N — 1 - zn) = 

Pr{2Re[(z N - v N )* • (r N - v N )] > in(zN,v N )} 

(7) 

where * denotes the complex conjugate and • de- 
notes the scalar product of two vectors. The 
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squared Euclidean distance between the transmit- 
ted and error path is given by 

d 2 (z N , V N ) = l Z i “ V i| 2 = d > ? 

1=1 i=l 

Since dj is equal to one of the Aj, 0 < j < M/2, 
(Fig. 1) d 2 (rN,ZN) can can be expressed as 

M/ 2-1 

d 2 (r N ,z N )= ^2 wjA 2 (8) 

i=0 

With infinite interleaving, the sequence aN of fading 
amplitudes constitutes an independent and iden- 
tically distributed sequence. Equation (7) can be 
rewritten as 

P(v N — ► zn) = Pr{2Re[(zN - v n)* * n^] > 
d 2 (z N , v N ) - 2Re[(z N - v N )* • (a N x v N - v N )]} 

T ( 9 ) 
Let us suppose for simplicity that a^ is real and the 

modulation scheme is M-PSK. Then the right hand 

side of Eq.(9) can be simplified by noting that: 

-2Re [(z N - v N )* • v N ] = d 2 (z N , v N ) 

and 


In this analysis, we shall distinguish two cases, de- 
pending on the value of the ECL 6 h- For large £ H , 
using the approximation Q(x) < l/2exp(-x 2 /2) for 
x > 1, P(v N — *• zjm ) can be bounded by or equiva- 
lently 

P(v N — z N ) = 

2<Td 1 m 2 d 4 \ 

d,D - 2 + ct 2D 46XP \ 2 4<r 2 d 2 + ct 2 D ) 

(13) 

with 

D = £d? (14) 

i=l 

The formula (13) can be used for codes with large 
ECL for any SNR or for codes with medium ECL 
for small SNR where the influence of error paths 
with high effective lengths is significant. The bit 
error probability is upperbounded by 

Pb < r VoPd.D (15) 

K d,D 

where b^D is the total number of erroneous infor- 
mation bits on all paths at the Euclidean distance 
d from the transmitted path characterized by the 
distance D and k is the number of information bits 
in a symbol. 


-2Re[(z N - v N )* ■ (a N X v N )] = 



In the second case, when <5 h is small, the Gaussian 
assumption is no longer valid and the probability 
has to be expressed as or equivalently 


Then Eq.(9) can be rewritten as 
P(v N -*■ z N ) = 


N 

Pr{2Re [(zn - vn)* • nN] > ^ a;d 2 } (10) 

i=l 

For a given realization of the fading variable aN, the 
term on the left hand side of (10) is a zero- mean 
Gaussian random variable with variance 4cr 2 d 2 and 
the term on the right hand side is a constant which 
depends on aN- The conditional pairwise error 
probability for a given realization of the fading ran- 
dom variable aN is given by 


P(V N 


zn|»n) = Q 


Eili M? 

2ad 


(ii) 


where Q(x) is the complementary error function de- 
fined by 



( 12 ) 


P(v N -»• z N ) = 


/ P(vn — *■ z N |aN)p(a!) . . . p(aN)dai . . . da N 

J «N 


For large signal-to-noise ratios, we can write 

p/ ) ^ 1 TT 8 a 2 d 2 

P(v N - z N ) < -JJ 


2 1=\ d ? 


(16) 


(17) 


or equivalently 

P(v N - «„) = P Aw < (18) 

It should be pointed out that the formula (18) is 
valid for short ECL codes and large SNR only. 


The bit error probability is obtained by using the 
union bound technique as 

Pb ^ j7 yi ,wPd ,w (19) 

d,W 
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where b^w is the total number of erroneous infor- 
mation bits on all paths characterized by the Eu- 
clidean distance d and the set w. 

Code Design Criteria 

The analytical results indicate that the design cri- 
teria for fading channels will depend on ECL val- 
ues. For the codes with large ECL, the error prob- 
ability (Eqs. (13)) is dominated by the minimum 
Euclidean distance, while for the codes with small 
ECL, Eq.(18) and Eq.(19 show that the error per- 
formance is mostly dependent on the ECL, the min- 
imum PD and the number of erroneous bits on error 
paths, bd . 

To construct good codes for fading channel, it is 
very important to avoid having parallel transitions 
in the trellis. All the previously known 16-PSK and 
16-QAM codes[6][5][7], however, have parallel tran- 
sitions. An upper bound on the maximum ECL has 
been obtained[8]. Table 1 shows the upper bounds 
on ECL, T m , for 16-ary trellis codes with constraint 
length v. 

According to the possible values of ECL, to min- 
imize error probability on fading channels one 
should attempt to 

• when the possible ECL is small 

(I-a) design the codes with maximum ECL and 
(I-b) among the codes obtained form (I-a) max- 
imize d p first, and then maximize d e 
(I-c) minimize N(d p ) and N(d e ) among the 
found codes 

• when the possible ECL is large 
(Il-a) maximize d e 

(II-b) minimize N(d e ) among the found codes 

The criterion set I can be used to construct the 
codes with small possible ECL. It is possible to 
construct new 16-PSK TCM codes with no paral- 
lel transitions, though the minimum Euclidean dis- 
tances are inferior to the known codes. 

4 Code Construction 

A computer search based on the defined criteria has 
been carried out in order to find out the best codes 
for fading channels. To minimize bit error rate, the 
Gray mapping (i.e., the neighboring symbols in a 
constellation differ only by one bit) is adopted in 
code construction. 


A stack algorithm for calculating the minimum free 
Hamming distance of convolutional codes has been 
used for calculating the minimum free Euclidean 
distance of trellis codes by replacing the Ham- 
ming weight of channel symbols by the Euclidean 
weight[l]. This algorithm is referred to as Algo- 
rithm 1 in this paper. 

A modified version of this algorithm, which enables 
to compute not only d e but also N(d e ), has been 
developed[9]. In the modified algorithm, additional 
information on the multiplicity of the merging paths 
to a state with the lowest Euclidean weight is added 
into the stack memory. The search does not termi- 
nate d e is reached, but continues till all the paths at 
d e are found. We refer to this modified algorithm 
as Algorithm 2 here. 

In order to compute d p , another modification of 
the previous algorithms has been made. The new 
version can compute the minimum PD among the 
paths of any specified effective length. We refer to 
the new algorithm which computes only d p as Algo- 
rithm 3. The other new algorithm which computes 
d p as well as N(d p ) is referred to as Algorithm 4. 

From Table 1, one can see that the maximum pos- 
sible ECLs for 16-ary trellis codes are not large. 
According to the code design criteria in Section 3, 
the computer search should be carried out following 
the criteria set I. 

To make the search efficient, Algorithms 3, 1, 4 and 
2 are used successively for searching the best codes 
in terms of maximum ECL, d p , d € , minimum N(d p ) 
and N(d e ) among the codes obtained at each pre- 
vious step. 

Table 2 lists the parameters of some new 16-PSK 
codes, where Code Fi is 2*-state code proposed for 
fading channel. As references, the parameters of 
the corresponding Ungerboeck 16-PSK TCM codes 
given in [6] are also listed as Code Ui. All of the 
other known 16-PSK codes[5][7] have parallel tran- 
sitions and therefore they behave similarly on fad- 
ing channels. In the table, the parity-check poly- 
nomials are given in the octal form, and N and G 
denote the natural mapping and the Gary mapping 
respectively. 

It can be seen from the table that the proposed 
codes for fading channels have the maximum ECL 
and maximum d p , but smaller d e than the corre- 
sponding Ui type codes. It should be noted that 
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the search for codes was only based on the first 
term of d p and d e . It is possible to achieve a slight 
improvement in error rate performance by optimiz- 
ing the distance spectrum though it requires more 
complicated search algorithms. 

5 Performance Evaluation 

The performance evaluation in terms of bit er- 
ror rate (BER) has been conducted. To compute 
the analytical bounds for the BER performance, a 
modified Viterbi algorithm has been implemented 
to compute distance spectrum. Concurrently, ex- 
tensive Monte Carlo computer simulations of the 
schemes above were carried out. 

Fig. 2 shows the analytical and simulation BER re- 
sults for the 16-state code F4 compared with code 
U4, on Rayleigh channel. Fig. 3 illustrates the re- 
sults for the 128-state code F7 compared with code 
U7. A summary of the performance of all proposed 
Fi type codes is presented in Fig. 4. 

The performance results show that all the Unger- 
boeck codes have error floors above BER=10 -4 on 
fading channel due to parallel transitions. The pro- 
posed codes achieve significant improvement. As 
the simulation results show, at the BER = 10“ 3 ( 
which is an acceptable rate for digital voice trans- 
mission), the difference between the two classes of 
codes varies from 1.5 dB for the 256-state codes 
to 6 dB for the 128-state codes on Rayleigh fad- 
ing channels. This difference is even larger at lower 
error rates. 

6 Conclusions 

The design criteria based on ECL and PD are ap- 
plied to construction of new 16-PSK trellis codes 
with improved error rate performance on fading 
channels. The code performance is evaluated both 
analytically and by simulation. The comparison 
with the previously known codes of the same com- 
plexity shows that the proposed codes can achieve 
significant improvement in bit error rate. The new 
16-PSK trellis codes will meet the requirements for 
high data rate transmission in future satellite com- 
munications. 
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Figure 1: Notation for 16-PSK Modulation 



Figure 2: BER performance of the 16-state 16PSK 

TCM codes on Rayleigh channel 

Dash: Analysis; Solid: Monte Carlo simulation 


Figure 3: BER performance of the 128-state 16PSK 

TCM codes on Rayleigh channel 

Dash: Analysis; Solid: Monte Carlo simulation 



Table 1 Upper Bound on Maximum ECL . 
v 2 3 4 5 6 7 8 9 10 11 ... 

L m 12223334 4 4 ... 


Figure 4: Summary of BER performance of 16PSK 
TCM codes on Rayleigh channel 
Dash 1: Uncoded 8-PSK; Dash 2 to 4: Codes U3,U7 
and U8; Solid 1 to 6: Codes F3,F4,F5,F6,F7 and 
F8; All results are obtained by Monte Carlo simu- 



Table 2 

16-PSK TCM 

codes , 


lations 


i.d. 



h 1 

hT 

Sh 

d e 

N(d e ) 

dp 

N(d p ) 

Map. 

U3 

- 

- 

04 

13 

1 

1.47 

4.00 

2.00 

2.00 

N 

F3 

15 

17 

05 

13 

2 

0.89 

0.19 

0.61 

0.25 

G 

U4 

- 

- 

04 

23 

1 

1.62 

8.00 

2.00 

2.00 

N 

F4 

37 

21 

15 

33 

2 

0.89 

0.34 

2.47 

0.50 

G 

U5 

- 

- 

10 

45 

1 

1.91 

8.00 

2.00 

2.00 

N 

F5 

75 

73 

47 

57 

2 

1.48 

0.77 

13.7 

1.00 

G 

U6 

- 

- 

024 

103 

1 

2.00 

2.00 

2.00 

2.00 

N 

F6 

165 

175 

115 

143 

3 

1.48 

1.90 

0.68 

1.00 

G 

U7 

- 

- 

024 

203 

1 

2.00 

2.00 

2.00 

2.00 

N 

F7 

337 

343 

307 

211 

3 

1.48 

0.03 

4.69 

2.00 

G 

U8 

- 

374 

176 

427 

1 

2.08 

7.88 

2.00 

1.00 

N 

F8* 

771 

513 

463 

405 

3 

1.78 

1.39 

27.3 

1.00 
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